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ABSTRACT 

Evaluators of argument are frequently confronted by 
conflicting claims. While these claims are usually based on 
probabilities, they are often resolved with the accepted claim 
treated as though it were "true," while the rejected claim is treated 
as though it were "false." Scenario testing is the label applied to a 
set of procedures by which competing claims are evaluated as 
"probabilistic" scenarios. Resolution of a proposition by scenario 
testing requires that an accounting of each substantiated scenario be 
made. Scenario testing begins with the assumption that conflicting 
claims should not be resolved by disjunctive choices. Rather, it 
assumes that descriptions and predictions provided by the debaters in 
a round are, by their nature, probabilistic. As a consequence, 
evaluation of a policy, whether the single policy offered by the 
Affirmative or the combined policies offered by the Affirmative and 
Negative are most reasonably evaluated when the probabilities of 
their descriptions and predictions are taken into account. An optimum 
policy choice, therefore, represents the efficacy of a policy (or 
policies) across the range of scenarios it must address. This 
procedure equalizes the burdens of the advocate and counter-advocate 
and provides a set of guidelines for evaluating their conflicting 
claims. (Author/HOD) 



******************************************************** 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 
*********************************************************************** 



i Ml 



i. 



00 



in 
o 

Ul 



Scenario-Testing: Decision Rules for 
Evaluating Conflicting Probabilistic Claims 



Craig A. Dudczak 
University of Oklahoma 



David Baker 
St, Mark's School of Texas 



Speech Communication Association 
1984 Conference, Chicago, Illinois 
November 2, 1984 



X 



U.S. DEPARTMENT OF EDUCATION 

NATIONAL INSTITUTE OF EDUCATION 
EDUCATIONAL RESOUHCES INFORMATION 
CENTER lERlCl 
This document has been reproduced as 
received from the person or organiMiion 
o'lginating it 

Minor changes have been made lo improve 
reproduction quality 

• Points of viokv or opinions stated m this rtocu 
ment do not imcossarily roproscnt official NIE 
po^ • ' policy 



ABSTRACT: 

Evaluators of argument are frequently confronted by conflicting claims. 
While these claims are usually based on probabilities, they are often resolved 
with the accepted claim treated as though it were "true," whiJLe the rejected 
claim is treated as though it were "false." Scenario-testing is the label 
applied to a set of procedures by which competing claims are evaluated as pro- 
babilistic scenarios, Resoluation of a proposition by scenario-testing requires 
that an accounting of each substantiated scenario be made» This paper develops 
an explanation of the scenario-testing procedure, its rationale, and the impli- 
cations of its use in argumentation* 
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Scenario-Testing: Decision Rules for 
Evaluating Conflicting Probabilistic Claims 

If there is a beginning point for this paper, it probably can be found in 
the evolution of arguments in recei>t policy topic debates. A few years ago 
there was a rare overlap between the High School and College resolutions. Many 
will recall the fairly "stock" Central American case which alleged the following: 

t 

1. Nicaragua is really a peace-loving nation. 

2. The Nicaraguan military build-up is a response to U.S. military 
support/arms sales to its neighbors. 

3. Curtailment of U.S. military intervention/arms sales to Central 
America would result in a negotiated settlement with Nicaragua. 

While the particulars would vary from case-to-case, the preceding outline was a 
typical case approach. An equally typical "stock" response to the Central Ameri- 
can case was often found in the following: 

1. Nicaragua is really a Cuban/Soviet puppet. 

2. The Nicaraguan military build-up is a prelude to regional political 
and military domination. 

3. Curtailment of U.S. military intervention/arms sales would deliver 
Central America to Nicaraguan domination. 

While this response also varied in case-by-case applications, it paralleled 
the sequence of claims made by the Affirmative advocate with fairly direct counter- 
claims. When confronted by these two disjunctive descriptions of reality in 
Central America, we, as debate critics, are forced to make a choice. If we accept 
the first description as true, we would almost invariably vote for the Affirmative. 
If we accepted the second, it would almost invariably result in a decision for the 
Negative. It has occurred to us, as it has undoubtedly occurred to others, that 
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the choice between these two descriptions of Central America both contained ele- 
ments which might be true. As such, making a disjunctive choice between two pro- 
babilistic representations, of reality is unsatisfactory. 

It is because we feel that resolving conflicting claims as though they were 
disjunctive choices is not a satisfying or realistic evaluation of issues that we 
offer this paper. We propose that conflicting descriptions be treated as scen- 
arios. In developing a perspective of treating conflicting claims as scenarios, 
we will offer an explanation of scenarios, examine their relevance as a metaphor 
for academic debate, and illustrate the implications of their use in academic de-" 
bate. We label the perspective we offer ''scenario-testing" because the accepta- 
bility of any single scenario necessarily includes an assessment of the probabil- 
ity of its determinant parts, and may conditionally consider its comparative abil- 
ity to withstand alternative scenarios. Finally, we articula^w a set of proce- 
dures for evaluating competitive scenarios. 

The Nature of Scenarios 

Scenario is a referent term which is most frequently found in fields associ- 
ated with statistical analysis (Clapham et al., 171) in social, political, military 
and economic sciences. Analytic procedures which often generate reference seen*- 
arios include operations analysis , operations research, systems engineering, manage 
ment science, and systems analysis (Hoos xl, 42). Our definition of a scenario 
is taken from Slovlc et al. (177) and stipulates that the essential characteristic 
of a scenario "consists of a series of events linked together in narrative form. 
Normatively, the probability of a multlevent's scenario's happening *s a multipli- 
cative function of the probability of the individual links." 

We think this definition suggests two functions of a scenario--one descrip- 
tive and the other predictive. The descriptive function of a scenario is to ex- 
plain the relationships among a sequence of events necessary to arrive at a con- 
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clusioiu The predictive function of a scenario addresses the question of how 
likely the scenario is to occur. To a large degree, description and prediction 
.are inseparable in a scenario. The prediction of each event in a multievent se- 
quence assumes its predecessor. The probability of each successive event allows 
the scenario description to continue to its conclusion. Nevertheless, separation 
of the descriptive and predictive functions of scenarios will allow us to better 
understand their dynamic interaction. 

• Scenarios as d escription . 

In its simplest representation, a scenario is a story, a narrative, a speci- 
fication of a sequence of events. In The T hird World War, a speculative account 
of the events which would lead to a Soviet invasion of Western Europe and the sub 
sequent progress of the war. General Sir John Hackett masterfully captures the 
story-telling aspect of a scenario — So masterfully, in fact, that his novel en- 
joyed considerable success in the commercial market. While most scenarios aren^t 
written for a general public audience, they nevertheless incorporate a narrative 
accounting of events. The narration gives coherence to the sequence of events » 
And the more carefully crafted the association among the events, the greater the 
bellevabillty of the predlction(s) made or implied by the scenario. But more 
about prediction later. 

The descriptive function of scenarios gives meaning to events by relating 
• them in time, hierarchy, sequence, salience, and the like. Of course, the Innoc- 
uous label of "description** is not bias-free, k description, any description, 
carries the distortions of perception carried by the one offering 'he description 
Tills is not even to assume an active bias on the part of the describer, for the 
distortions may equally be accounted for through the symbol system (Burke 44-62) 
or within the cognitive heuristics of the social actor (Tversky & Kahnemann, 
"Availability" 207-232; "Heuristics and biases" 1124-1131). 
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But an acknowledgment of bias in description is not to suggest an objective 
standard against which distortion can be measured. It has been argued by Schneider 
et al,, (204-223) that our perceptual distortions only can be described in 
terms of their direction, but not in terms of their degree. Certainly, between 
symbolic interaction and constructivist notions of symbolic and phenomenal real- 
ity, we should not expect an objective measuring stick. 

The point we wish to extract here is that the descriptive function of the 
scenario is subject to systematic distortion. As such, we offer a partial list- 
ing of the sources of bias which may infuse scenarios: 

First, a scenario description is infused with a set of values, assumptions, 
and goals. Consider the primary differences between the two Central American 
scenarios offered in the introduction to this paper. The person who identifies 
with the first scenario (Nicaragua is really peace-loving, etc.) may be incapable 
of recognizing the second description as anything other than the "rantings of a 
neo-colonial imperialistic war monger." Of course the hyperbole is interchange- 
able. The values, assumptions, and goals held within a scenario represent a pre- 
commitment to instruments which become self-validating (Clapham et al., 171). 

What frequently happens i's that the perceiver, already wedded to the assump- 
tions, goals, and values of a description becomes selectively biased in recogni- 
tion of confirming instances and selectively ignorant of those disconf irming in- 
stances. As obvious as this caution may seem to be, Slovic et al. , (176) warn 
that even those experts trained in analytic techniques are insensitive to their 
own biases. 

On a similar vein, a self-fulfilling prophecy further creates a false sense 
of the validity of the assumptions initially accepted. As Jouvenal notes, "any 
so-called 'prediction' is always a starting point for examination of what should 
be done on the assumption that it is true, but also always an outcome of assump- 
tions concerning what will have to be done to mnke it come true." (120) 
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A second source of bias in scenario description may be a tendency toward 
simplification of the description due to cognitive limitations (Slovic et al., 
168-169). The number of variables and the complexity of their inter-relation- 
ships itself may make a simplification expedient, if not necessary. It does ap- 
pear, however, that there tends to be a preference towards describing model scen- 
arios in terms of those variables which are subject to quantification (Hoos 127, 
133; Ascher, 253-254) at the expense of those variables which are subjective, in- 
tangible, and unquantif iable. Certain categories of variables, as Hoos notes, 
are thus excluded from active consideration in the model: 

While mention is made of "political, sociological, and cultural 
variables," such factors have proven elusive of capture and con- 
trol by the "systematic" approach. Consequently, they have been 
omitted. Only those variables which could be handled quantita- 
tively have been taken into account, and the accounting has re- 
mained largely economic. (78) 

While description of social, political, and cultural variables are possible 
without their quantification in a scenario, their habitual exclusion, and per- 
haps the exclusion of other categories of unquantif iable variables, represents a 
systematic bias in scenario construction. 

A third source of bias in scenario description is the bias in £avor of the 
technical solution in preference to its alternatives. Part of the reason for this 
bias may be that the very act of constructing and testing scenarios tends to be 
performed, by modelers who are more skilled in the construction of their models 
then they are in understanding the dynamics of the context in which the variables 
exist (Starr 1232; Ascher 254, 258). This is to say that the scenarios become 
abstracted from the events they purport to represent and they become reified in 
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in that they may bo treated as a concrete object regardless of the correspondence 
with the events which gave them life* But the more important issue here is that 
a technical representation of an event may ignore a more commonplace understand- 
ing of the dynamics of a scenario. The tension between "expert" technical repre- 
sentations of scenarios with their assumptions of "normative" operations may con- 
flict with non-expert representations of the same event. Technical people may 
accuse the public of neglecting facts and irrationality. Non-technical people 
may be equally prone to view technical solutions are esoteric and frivolous. 
Each side may be equally guilty of implying a universal rationality to their re- 
spective positions without ever bothering to articulate their base assumptions or 
hold them to challenge. (Otway & Winterfeldt 250) Since it is the technical model 
builders who are more likely to construct the descriptions, we probably need to 
be most sensitive to their tendencies toward reification of their models: 

Construction of abstract models intended to describe in mathema«- 
tical terms the complex interralationships governing the process 
of economic growth has become one of the favorite occupations of 
economic theorists. Unfortunately, the lack of factual knowledge 
of conditions existing In the real world forces the model builder 
to base many, if not all, of his general conclusions on all kinds 
of a priori assumptions, chosen for their convenience rather than 
for their corresjTondence to observed facts. (Leontief 32) 

This technical bias of model construction barkens back to the original qual- 
ifier on the use of scenarios—namely, that the assumptions, values, and goals 
of a scenario are never validated through the model if the model itself becomes 
tautologies A scenario informs us of its assumptions which may be described and 
tested. Verification of the assumptions cannot validate the model because it 
can't inform us of what we haven't tested. At best, an attempt at verification 
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can only disconfirm assumptions. So that the ultimate test of a scenario is under- 
stood to occur when the assumptions and values are subject to critique outside 
the framework of the model. It is only when scenarios are held up against exter- 
nal criteria can their validity be verified, if only indirectly (Feyerband 33,38; 
Clapham et al., 173; Slovic et al. , 178). 

The descriptive function of the scenario gives the scenario plausibility and 

coherence. The predictive function of a scenario is what informs us of its prob- 

« 

ability. In this next section we will consider elements which determine a seen- 
ario's predictive ability. 

2 . Scenario as prediction 

We will use the commonplace term "prediction" in a somewhat specialized way. 
Normally, we think of the term prediction as referring to future events, as in 
the statement, "If the United States curtails military -assistance to Central Amer- 
ica, Nicaragua will negotiate." However, there is also a retrospective sense in 
which we use predictions that is applicable to scenario construction. So, for 
instance, when one argues that a past event occurred for a particular reason — i.e., 
the Civil war was caused by slavery — one is engaging in this type of retrospection. 

Scenarios make predictions to the extent that their interdependent parts are 
probabilistic. The scenario is the aggregation of the probabilities of the indi- 
vidual parts. As such, there are a number of concerns about the calculation of 
^ probabilities which need to considered when examining the predicti^ve function of 
a scenario. 

The first consideration, and we think one of the most important, is whether 
the predictive power of a scenario is primarily based upon objective or subjective 
probabilities. An objective probability, otherwise known as mathematical proba- 
bility (Rieke & Sillars 21-26) is based upon empirical measures of verifiable 
events. Hence, the coin flip dictating chance occurrence of 50-50 for either 
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heads or tails is based on observation and measurement which can be confirmed in- 
dependently of the particular observer. We will comment about the forecasts de- 
rived from empirical events, shortly. But for here suffice it to say that the 
objective/mathematical realm of probability is empirically verifiable. 

The Subjective realm of probability is ultimately based upon the strength 
of belief a person holds (Sjoberg 39). While a subjective probability may be 

informed by objective data, it is not dependent upon empirical verification. 

« 

Perhaps an example here would clarify the distinction between objective and sub- 
jective probabilities. There is data which indicates that flying is about four 
times safer than auto transportation per passenger mile travelled. This object- 
ive data is empirically derived from the calculations of number of persons who 
avail themselves to each mode of transportation div^r^'^.d bv the number of fatali- 
ties within each mode. Nevertheless, there are those individuals who have an 
abject fear of flying, and even when informed of the comparative data of safety 
for flying over driving, will prefer not to fly. This fear of flying may be taken 
to represent the "strength of belief" associated with subjective probability. 

While all probabilities will be in some sense subjective, we believe it is 
important to distingui'sh between objective and subjective dimensions of probabil- 
ity. We think that the labels "probability" and "prediction" are frequently used 
in what Rieke and Sillars had called a "pseudo-logical" form. (211) Just the labe 
"more than 50% probable" implie."^ a mathematical objectivity. Unfortunately, how- 
ever, we feel that mathematically precise statements are frequently offered with- 
out the benefit of corresponding empirical justification. We frequently offer 
such mathematical, pseudo-logical statements not only not having calculated a mathe- 
matical basis for our precise statements, these are frequently claims which are 
not capable of heving a mathematically precise base. When the Union of Concerned 
Scientists advances the minute hand on its atomic clock closer to midnight, it 
connotes the impression of an objective calculation. In point of argument, such 
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exercises, while symbolically powerful, are empirically void of verification* 
Sjoberg reminds us of the importance of disLinguishing between probabilities 
which are empirically grounded from those which are not: 

All probabilities are in some sense subjective. However, from 
a pragmatic point of view it is clearly important to distinguish 
between the belief strength that a certain person holds and the 
belief strength that is justified by mt^ans of some theoretical 
and empirical considerations. (39-40) 

Having emphasized the difference between objective and subjective probabil- 
ities, the subsequent problems one encounters in making predictions through scen- 
arios will indicate which type of probability underlies the problem. Again, we 
acknowledge the interrelation between the two types, but this does not make them 
the same* The next several limitations will refer to problems with subjective 
probabilities. 

Probably the most important consideration in subjective probabilities is the 
question of what factors might exagerate the strength of belief in excess of the 
grounds to support the belief. Sjoberg (39-57) articulates a set of variables 
which will influence the subjective probability reported. He notes that be;,iefs 
are first focused on what arc perceived to be significant events* Beliefs are 
also clustered, may be based on sensory or perceptual data, as well as upon con- 
ceptually-based (abstract) information. Further, he notes that strength of belief 
is tied to emotion. Under strong needs, belief distortion is more likely. Cog- 
nitive systems, which underpin beliefs, may weaken under conditions of stress and 
create a need-aversive bias. 

It is easy to view these variables as existing in individuals. The question, 
however, is whether they also influence collective or Institutional beliefs. We 
think that there is substantial evidence to support the claim that popular myths, 
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legends^ and fairy tales abound in our world which havo denionsCrat;ed the potency 
of subjectivt^ly formulated beliefs. Om^ example, cited by Paul Watxlawick in l[ow 
Real if^ R; >^ 1 ? » tells of a popular belief in Seattle when it was reportiul that more 
windshield "pithing" was occurring. Suddenly, more and more instances of the 
windshield pithing seemed to confirm popular notions that atmospheric conditions 
had changed or testing had caused the observed phenomenon. In point of fact, the 
mystery was resolved when it was determined that the heightened state of peopleV* 
awaren^ass caused them to look more closely at their windshields (and from the out- 
side looking in, rather than from the inside looking out), and hence, report more 
instances of the phenomenon, Rep or tinges of windshield pithing had increased* 
Actual instances remained constant* 

One consequence of reliance upon subjective probability is a tendency toward 
over confidence in accepting the belief (Sjoberg 45). A test for overconf idence 
has been suggested by Schaefer et al., (331) whereby the strength of any single 
belief is placed in context of its surrounding web of beliefs. Where a particu- 
lar belief is found to be consistent with other beliefs surrounding a proposition. 
The more extensive the belief structure, the greater the confidence in the parti- 
cuiar belief in question. Bunn (131, 132) systematizes this procedure by arguing 
that subjective beliefs are best adhered to when the result of utilizing the full 
amount of information available. He notes: 

• . . the underlying resolve of the decision analyst is to util- 
ize the optimal amount of information. Furthermore, the p,eneral- 
ized consistency requirement of a subjective probability used in 
decision analysis should reflect consistency with the totality of 
evidence and beliefs held by the individual at that time. (131) 

In summary, subjective probabilities may be presented in a form which is 
indistinguishable from objective probabilities. They aBsume s ik;nif icance as 
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pseudo- logical statements which bear the terminology of objective probabilities, 
i.e., "probability, prediction," etc., but they do not encompass the empirical 
method which distinguishes objective probabilities. As a consequence, a sub- 
jectively stated probability creates a special problem for evaluation. If we 
choose to evaluate the likelihood of some future event based upon the strength 
of an advocates belief ^ ve need to Inquire about the basis of that belief , and 
test it against other beliefs whicn confirm it and which may themselves be^ sub- 
ject to some empirical verif ication» 

The third consideration to be made about the function of prediction in 
scenario construction addresses the objective category of probability. We should 
note that the context for which we consider objective probabilities often includes 
the terms "risk analysis," "forecasting," most types of "modeling," and "trend 
analysis*" Wliile these terms are not entirely interchangeable, they frequently 
display the characteristics of objective probabilities which is now addressed. 

Probably the greatest problem with objective probabilities rests in the 
tendency by the user, even when trained in statistical techniques, to confuse the 
multiplication of independent events with their averaging. (Tversky & Kahneman 
105-110). When defining a scenario, a prediction is often the conclusion of a 
sequence of predicate events, each of which contains a probability. The total 
probability of the event's occurrence, therefore, is the aggregation of each In- 
dependent link. As Slovic et al., (177) note, "the more links there are in a 
scenario, the lower the probability of the entire scenario's occurrence. The 
probability of the weakest link sets the upper limit on the probability of the 
entire narrative," However, there is evidence that this routine is violated. 

Human judges do not appear to evaluate scenarios accoding to these 
normative rules. We have begum to collect data suggesting that the 
probability of a multilink scenario is judged on the basis of the 
average likelihood of all of its links. Subsequent strong links 
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appear to "even out" or compensate for earlier weaker links, making 
it possible to construct scenarios with perceived probabilities 
that increase as they become longer, more detailed, and i.ormatively 
less probable. (Slovic et al., 177-178) 

Wtiile we will address this parallel further in the application of scenario- 
testing to academic debate, this problem appears to frequently beset the debate 
judge in his/her evaluation of multilink arguments. Rather than treating the 
probability of a particular consequence as the multiplicative probability of its 
predicate statements, debate judges may also engage in this type of "averaging" 
of probability to determine likelihood of a given claim. 

Additional problems which accompany the calculation of objective probabili- 
ties are tendencies to underestimate the error and unreliability inherent in 
small samples of data (Slovic et al., 169), generalizing model assumptions of 
risk to social acceptability without checking the actual social acceptability of 
the risk (Starr 1237; Sjoberg 51,52), and undue optimism in calculating benefit 
to cost estimates (Pouliquen 2). 

When aplied to trend analysis, the fundamental problem which exists for 
prediction is the assumption that empirical instances which have been validated 
from past experience will be recurrent in the future (Ascher 259; Hoos 131). 
Risk analysis, in addition to carrying the pseudo -logical ambiguity of the use 
•of the term "risk," (i.e., corresponding to subjective probability, the label 
"risk" implies a calculable prediction, which quite frequently does not exist), 
there is an additional concern added concerning the social acceptability of 
of risk. Otway and Winterfeldt (252-253) categorize a number of social risk 
variables which influence whether a particular risk analysis will be acceptable 
including whether the risk is voluntary, the personal control over the outcome 
of risk exposure, uncertainty of the consequences of the risk exposure, lack of 
personal experience with the risk, etc. 
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Applications of forec£.stlng to prediction also suffer from the absence of 
a general theory of forecasting, which often renders predictions as an ad hoc 
process (Bunn 127). Attempts to apply the most rigorous of models, often con- 
ceived to be applications of econometric modeling, despite their adaptation to 
new events and increasing sophistication of design, do not appreciably increase 
forecast accuracy, and In any event, do not show consistent results when measured 
against subsequent events appreciably greater than subjective forecasts. (Ascher 
256-257). 

One systematic problem with forecasts is their drop-off in accuracy as a 
function of time* As forecasts are extended into the future, they bear a linear 
i.e) -iuicnship to their error term such that the error factor generally doubles 
v/lth the passage of each increment of time (Ascher 258-259). 

Of course, the statistical products are further distorted by judgmental 
(subjective) variables which enter into their interpretation. Statisticians are 
biased in their interprecation of their own data (Sjoberg 48), and extreme odds 
and small probabilities are equally problematic in being assessed both in the 
opsration of a forecast model, as well in their subsequent interpretation (Sjo- 
berg 49). 

Even when trained in the use of statistical techniques, estimates of the 
probability of the occurrence of a future event are distorted by a tendency to 
ignore base rate Information (Slovlc et al., 165). An example of this would be 
the case where we know that the expected probability of a heads coming up In a 
trial of ten coin flips is one half (.50). When a trial yields only three heads, 
persons seem to be more influenced by the results of the immediate tibial and 
tend to discount the predicted outcome of 50%. This constitutes ignoring the 
base rate information. 

The consequence of these sources oi error and bias has led Slovic et al. 
(165) to conclude that many predictions are often based upon a greater amount 
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of Intuitive judgment than would be warranted given the empirical examination 
of the data available. The quality of the intuition sets the actual limit to 
the quality of the entire decision making process • 

Hence, we see that scenarios, while incorporating both descriptive and 
predictive functions , suffer a set of theoretical and methodological limitations 
which are frequently masked by the appearance of statistical procedures. Never- 
theless, scenarios represent an analytic procedure which has merit in two import- 
ant regards • 

First, because scenarios provide a narrative explanation of the sequence 
of events necessary to link events separated in time, they provide an explana- 
tory power which is otherwise absent. And second, because scenarios acterapt to 
provide a level of confidence of the probability of the occurence of the event, 
they have predictive power. 

They are used as a tool in various analytic and decision making procedures 
because they specify their assumptions and attempt to translate these assumptions 
to social values weighted by the probability of their occurrence. As such, we 
believe they can constitute a useful metaphor for the evaluation of conflicting 
claims often found in academic debate. 

Scenarios Applied to Debate 
It is our contention that the rationale for accepting a claim in debate 
is the functional equivalent of accepting a scenario* In applying systems • 
analysis to academic debate. Brock et al., (50-52) described a procedure which 
is fundamentally the construction of a scenario. Others have employed the 
procedures of systems analysis'*" in extending the two primary components of 
scenario construction— description of the system and evaluation of its predicted 
consequences. In contemporary debate, the resolution of policy choice will 
ultimately depend upon the adequacy of the description of component events in 
their dynamic interaction assessed against a set of goals specified for their 
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operation. When we take a case like the Central American case illustrated ear- 
lier in the introduction, we have a clear illustration of conflicting values and 
descriptions, as well as predicted consequences, confronting the decision maker 
in debate. 

Standard procedures for the resolution of these conflicts often focuses 
upon default conditions. That is, who neglected to extend the third subpoint of 
the topicality argument, or was dropped the impact arguments of the first disad- 
vantage, often becomes the standard for resolving the debate. Debate critics 
look for the least common denominator when rendering these types of decisons. 

But in the case where conflicting scenarios, ala the two descriptions of 
Central America, contain antithetic assumptions, values, and predictive conse- 
quences, there is a gnawing sense of unreality. A vote for the Affirmative which 
is predicated upon the "truth" of their description and predictions assumes the 
"best case scenario." True, and only true, of it actually constitutes so high 
a level of probability as to render any alternative scenario implausible. 

We think it is contingent upon the debaters to establish the support for 
their claims, and where this support results in the indeterminability of one 
scenario over its alternate, it would be more rationale to resolve the dispute 
by means other than the disjunctive choice we believe is exercised in decision 
making in academic debate. 

The more rational approach, we contend, would be to evaluate the efficacy 
of a policy across the likely range of probable scenarios. We return to the 
Central American case to illustrate this point. We have already stated that a 
decision for the Affirmative policy of curtailing U.S. intervention/arms sales 
in Central America is justified if, and only if, the Affirmative scenario is 
substantially superior (more probable) than any of its alternatives. In the 
case where the Negative scenario(s) is/are evaluated as equal or greater in its/ 
their likelihood of occurrence than the Affirmative's, a decision of the optimum 
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case must include consideration of the worst case scenario in addition to any 
other, So foi Central America, the policy choice to curtail U.S. intervention/ 
arms salens would be sub-optimal, and probably counterproductive, in the case 
where the Negative scenario was probable. 

The circumstance where no single scenario demonstrates superiority to 
its alternative (s) implies that the decision rule guiding selection of policy 
should be in favor of the policy which best accomodates the range of scenarios. 
We think this procedure would generally tend to favor policies oriented coward 
incrementalism (Pfau), We use the example of the bridge-builder to illustrate 
this opinion. 

Let us assume that a bridge were to be constructed across a river. Let 
us further assume that different structural characteristics of types of bridges 
were suited to different capacity. In the case were type A bridge was optimal 
for a low traffic flow (let's say 5,000 cars per day), but sub-optimal for heavy 
traffic flow (let's say 15,000 cars per day), and type B bridge were the opposite 
in its characteristics (optimal for heavy flow, and sub-optimal for low flow), 
then the decsion for which type of bridge to build would depend upon the average 
capacity of each type of bridge to accomodate the projected traffic flows repre- 
sented by each scenario. Of course, a theoretic type C bridge which was neither 
optimal for the extremes, but best suited across the range of estimates for traf- 
fic flow, would be optimal across the rabge of likely scenarios. 

This is the representative example to the argument we are making. The best 
policy is the one which is best capable of accomodating the range of scenarios 
which are likely to occur. It rejects the case were an Affirmative tailors its 
solvency to assuming, only its optimum scenario as much as it rejects the Negative 
policy which is optimal only under a single scenario. Of course, these types of 
comparisons may occur in debate, in which case, the decision rule is for the cri- 
tic to apply the best-case and worst case scenario to each respective policy. 
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We believe there are at least two types of cases where the scenario-testing 
procedures may be applied. These represent the lnf;tance where the Negative does 
not defend an alternative policy as one type of case^ with the other type being 
the instance where the Negative offers one or more alternative policies. (We 
. will refrain from commenting on the desirability of single versus multiple alter- 

m 

native policies.) 

In the case where the negative refrains from offering an alternative policy, 
we believe that the Affirmative scenario may be tested by challenging its descrip- 
tion and prediction. We would support the position that a scenario represents 
a justification for action being taken (Dudczak 232-235). Absent the predicate 
for action^ there is no requirement upon the Negative to offer an alternative 
scenario or policy which addresses the Affirmative scenario. In the case of 
Central America « the failure to substantiate the Affirmative scenario as probable, 
either through the demonstration of its inappropriate description or inadequate 
prediction, would serve as the logical I'allure to warrant the Affirmative's 
course of action. Because only a single scenario is presented does not establish 
its substantive probative value. It may be independently tested by criteria we 
have already articulated in earlier sections of this paper. 

Of course, strategically, the Negative may elect to offer an alternative 
policy either predicated upon the Affirmative's scenario or its own scenario* 
If measured against the Affirmative's scenario, then the Negative policy would 
need to demonstrate it represents a more optimum response to the scenario than 
the Affirmative's policy. In the case were competing scenarios were not demon- 
strably superior to each other, then the decision rule would be to assess the 
competing policies across the range of scenarios offered. 

To articulate the decision rules implied by scenario-testing, we would offer 
the following as a tentative set of guidelines to be applied to resolve con- 
flicting claims: 
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. When assessing a single policy (Affirmative) against a single scenario, 

the following guidelines to resolving issues would be appropriate: 
A. The policy would be evaluated by its predictive ability to obtain the 
values and goals implied or stated in the scenario. 

1. Evaluation of the probability of the scenario is a substantive issue 
to be resolved by the support offered for it. As part of its support 
is its ability to withstand challenge to the description or prediction 
of its subordinate parts. 

2. The probability of a scenario is the multiplicative product of each of 
its dependent events. (The probability of the consequence is determined 
by multiplying the probability of each necessary event leading to the con- 
clusion.) 

3. The upper limit to the probability of the total scenario is set by the 
least probabilistic event of its dependent parts. (An event which has 
no demonstrable propability is assigned a probability of 0.0, which 
means that the conclusion is void.) 

4. The assessment of subjective probabilities within the scenario is deter- 
mined by its consistency with other associated beliefs which f.re grounded 
in some empirically verifiable events. 

5. Prediction assumes correspondence between the mechanisms assumed by the 
evidence with the mechanisms provided by the proposal. Brock et al», (98) 
stipulate that such predictions normally consist of demonstration through 
pilot programs or through analogy with other, similar programs. 

B. Descriptions of a scenario are subject to tests of coherence and consistency. 
1. Coherence refers to the sequence of events leading to some conclusion 
which rationally (logically) follows from the sequence. (This postulate 
means that while there may be alternate routes to a particular conclusion 
(Bertalanffy lA), the advocate must provide the description of events ne- 
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cessary to arrive at the conclusion. 
2. Consistency refers to unequivocal uses of language such that there Is 
fidelity between the sense in which terms are used. Willard (5) goes 
further to say that prediction ultimately depends upon a consistent por- 
trayal of reality. 

2. When assessing a single policy (Affirmative) against conflicting scenarios, 
the preceding guidelines would be used with the addition of the following: 

C. The policy should be evaluated against all scenarios which equal or exceed 
the referent (Affirmative) scenario. (The Affirmative policy is evaluated 
across the range of scenarios , including its own, as well as any scenarios 
hostile to it.) 

1. The weighting of the optimum scenario shall be no greater than any seen*- 
ario demonstrated to be equal in its probability. 

2. The default condition if the consequences of the adverse (worst case) 
scenario is greater than the optimum (best case) scenario would be to 
suspend judgment on taking any action. 

3. When assessing multiple policies (Affirmative and Negative) against a single 
scenario (Affirmative) one would use the criteria cited under *'A" and '*B" 
plus the following guidelines: (Negative accepts Affirmative description) 

D. Both policies would be evaluated for their effects (both positive and nega-* 
tive) towards attaining the goals stipulated by the scenario. 

1. Benefits would be evaluated by probability of effects, costs of imple- 
mentation, and time frame for implementation through normal processes of 
the mechanisms offered for each policy. 

2. Costs are evaluated as negative benefits and are incorporated into the 
the equation of the single scenario (Nagel & Neef 405). 

4. When assessing multiple policies (Affinnative and Negative) against multiple 
scenarios (Affirmative and Negative), one would use all of the preceding 
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guidelines (''A'* - "D'*) plus the following: 

E. Each policy would bo evaluated across the set of competing scenarios which 
have a probability equal to or exceeding the referent (Affirmative's) scen- 
ario: 

1, The optimum policy would be the one which demonstrated the best effects 
at the lowest costs across the range of alternatives, (Neither policy 
could be evaluated in a void by assuming only its own *'best case** set 
of circumstances, 

2, Negative benefits (costs) within the frame of one scenario may be eval- 
uated as benefits within the frame of reference of the optimum scenario. 
(For Instance ^ the '*cost** of intervention in the first Central American 
scenario might be evaluated as a ''benefit'* under the second scenario. 

5, Assumptions about the responsibility of advocates to identify and sustain 
their positions. 

F. While perhaps it goes without saying » the preceding guidelines are potent- 
ial ipsues to be identified and applied by the advocates within the debate. 
A judge should refrain from making evaluations within or across scenarios 
when not supplied wlrh reasons from the contending sides. 

G. The evaluation of probabilities ^ describtlons , and other substantive issues 
in the debate assume non-intervention on the part of the judge. Neverthe- 
less, the meaning fulness of any argument or issue to the debate requires 
that the advocates explain and support positions with evidence and analysis 
appropriate to their contentions. 

While these guidelines may not be exhaustive to every contingincy which may 
arise within an adversarial process .in debate, they represent a set of decision 
criteria which may be equitably applied to advocates on both sides of a proposi- 
tion. Some of the implications from their application are discussed in the the 
closing section. 
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Implications of, Scenar to- tes t^iiig 

While we are tempted to discuss the implications of scenario-testing at 
length, we will briefly address the implications of scenario-testing as a de- 
cison rule to several contemporary perspectives in debate. We believe that the 
primary implication for scenario testing is that it assigns a decision proce- 
dure to resolving the issues of debate when they are in conflict • We do not 
claim scenario-testing as a new paradigm, but rather as an evaluative reference 
for resolving dispute across paradigms* 

Himes (4) has already noted that the nature of paradigms does more to 
stipulate a way of viewing debate rather than providing the critic with a rule 
structure or framework for evaluating the issues of a debate. They douM^ pro** 
vide a framework by which we may adjudicate the issues of a debate round. We 
believe we have already provided a rationale for considering scenarios as an 
appropriate metaphor for viewing debate. Let us now attempt to distinguish 
the characteristics of scenarios-testing from several prevailing perspectives 
and practices in debate. 

1 • Scenario - testing as distinctive from hypothesis testing 

Probably the single greatest difference between scenario-testing and hypo- 
thesis-testing deals with its treatment of presumption. Hypothesis-testing places 
presumption against the proposition being advocated (Zarefsky 432). Scenario- 
testing does not place presumption in favor or in opposition to either scenario 
being advocated. Presumption is a moot issue for scenario-testing. This is not 
to deny the types of presumption which may inhere in certain categories of Issues 
which may be introduced into the debate (Whately 112-132). However, since the 
probability of any given scenario is determined without reference to a preference 
for the policy with which it represents the "best case," scenario-testing could 
be said to reject hypothesis-testing's perspective on presumption. This bears 
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at least some peripheral reference to conditional arguments as well. As It Is 
practiced, conditional positions are frequently offered without careful consid- 
eration of their implications across the scenario or scenarios present in the 
debate (Hollihan 175-178). 6ut if a conditional position is to be offered to 
that presented by the Affirmative, then it bears the responsibility of evalua- 
tion by the scenario or competing scenario offered for the debate « 

Applying the decision rules we have articulated in the preceding, it would 
be our contention that conditional p^'l-'cies would be poorly suited to demonstrate 

4 

their superiority across the range c 2 scenarios. Without development, a 
conditional argument would be hard put to demonstrate its relative superiority 
when assessed by the same criteria applied to the Affirmative's policy. 

2 . Scenario-testing as distinctive from policy-making * 

On face value it would appear that scenario-testing accepts many of the 
same procedures which are advocated for policy making and its attendent reli- 
ance on systems analsysi and cost-benefit analysis. We would accept that the 
areas of overlap are many. But as with our distinction from hypothesis-testing, 
we would maintain that the treatment of presumption distinguishes scenario- 
testing from policy making. Policy making begins with the notion that .hange 
is inevitable, and the best policy is that policy which best adapts to change 
(Brock et al., 153; Lichtman & Rohrer 239-240). 

This perspective assumes that the description of change offered by the 
Affirmative is correspondent with the nature of change in the system. Of course, 
an alternate scenario is an argument which at one or more levels disputes the 
description of the system, and with it its attendent assumptions about the nature 
of change. When the Negative offers an alternative description of the system 
with its prediction of events, it is contesting the assumption of the Affirmative 
description. If there is a residual presumption for the Affirmative policy ♦ it 
is only within the context of its own scenario. When it fails to sustain it,s 
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as Che only likely description of reality, it forfeits its presumption in favor 
of the change contingent upon its description. It may conceivably demonstrate 
that its policv is optimal across the range of scenarios, but it should not bu 
presumed to be ab.Ve to do so. 

Of course, we have earlier taken the position that even without a competing 
scenario there are grounds for rejecting the presumption of change for a proposi 
tion as necessarily corresponding with the particular change advocated by an 
Affirmative (Dudczak 233) , but the decision guidelines we have outlined would 
offer additional reason reason for rejecting this position, 

3, Scenario- testing as distinctiva from counter-warrants . 

Unlike counter-warrants which are grounded in the consequences of the reso- 
lution as opposed to the implications of a particular policy (Paulsen & Rhodes 
205-206) scenario-testing tests the consequences of a policy against an alter- 
native set of descriptions in the evaluation of a policy's benefits and costs. 
When a policy is evaluated against the description of a single scenario, one of 
the strategies available to the Negative is the assessments of the policy's 
"negative benefits" which may be based upon its "remote consequences." (Hoos 
xviii-xix) 

However, the strategy implicit in advancing an alternate scenario is to 
test the Affirmative policy outside of its own assumptions, which we believe are 
usually selected because they represent the "best case" scenario for the, Affirma 
tive, An Affirmative which describes its case in narrow terms (Brock 123), does 
so to minimize the likelihood of negative consequences. However, when the Nega- 
tive reframes the debate in terms of a broader description or one which is anti- 
thetical to the policy's "best case" scenario, it makes the Affirmative burden 
greater because the policy is loss likely to be adaptive outside of its narrow 
interpretation. (Brock et al., 12A) . Hence, scenario-testing does not Lequire 
consideration of alternate policies which are possible under the Affirmative's 
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scenario as a means of testing the resolution. Rather, scenario-testing allows 
the Negative to redefine the circumstances against which the Affirmative's (and 
if it wishes, the Negative Vo) policy should be considered 

Conclusi ons 

Scenario-testing begins with the assumption that conflicting claims should 
not be resolved by disjunctive choices. Rather, it assumes that descriptions 
and predictions provided by the debaters in a round are, by their nature, pro- 
babilistic. As a consequence, evaluation of policy, whether the single policy 
offered by the Affirmative or the combined policies offered by the Affirmative 
and Negative are most reasonably evaluated when the probabilities of their 
descriptions and predictions are taken into account. An optimum policy choice, 
therefore, represents the efficacy of a policy (or policies) across the range 
of likely scenarios it must address. This procedure equalizes the burdens of 
the advocate and counter*-advocate and provides a set of decision guidelines for 
evaluating their conflicting claims. 



2 a 



Scenario-testing, 25 



Notes 

"^One should be especially aware of the extensive writing of Lichtman and 
Rohrer on this issue. Brock et al . , probably deserve credit for the initial 
discussion and their text, Public Policy Decision - Making ; Systems Analysis 
and Comparative Advantages Debate , probably contains the most comprehensive 
treatment of the implications of systems analysis as it pertains to debate. 
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